Abstraction in decision-makers with limited information processing capabilities

A distinctive property of human and animal intelligence is the ability to form abstractions by neglecting irrelevant information which allows to separate structure from noise. From an information theoretic point of view abstractions are desirable because they allow for very efficient information processing. In artificial systems abstractions are often implemented through computationally costly formations of groups or clusters. In this work we establish the relation between the free-energy framework for decision making and rate-distortion theory and demonstrate how the application of rate-distortion for decision-making leads to the emergence of abstractions. We argue that abstractions are induced due to a limit in information processing capacity.Comment: Presented at the NIPS 2013 Workshop on Planning with Information Constraint

An information-theoretic on-line update principle for perception-action coupling

Inspired by findings of sensorimotor coupling in humans and animals, there has recently been a growing interest in the interaction between action and perception in robotic systems [Bogh et al., 2016]. Here we consider perception and action as two serial information channels with limited information-processing capacity. We follow [Genewein et al., 2015] and formulate a constrained optimization problem that maximizes utility under limited information-processing capacity in the two channels. As a solution we obtain an optimal perceptual channel and an optimal action channel that are coupled such that perceptual information is optimized with respect to downstream processing in the action module. The main novelty of this study is that we propose an online optimization procedure to find bounded-optimal perception and action channels in parameterized serial perception-action systems. In particular, we implement the perceptual channel as a multi-layer neural network and the action channel as a multinomial distribution. We illustrate our method in a NAO robot simulator with a simplified cup lifting task.Comment: 8 pages, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS

A Nonparametric Conjugate Prior Distribution for the Maximizing Argument of a Noisy Function

We propose a novel Bayesian approach to solve stochastic optimization problems that involve finding extrema of noisy, nonlinear functions. Previous work has focused on representing possible functions explicitly, which leads to a two-step procedure of first, doing inference over the function space and second, finding the extrema of these functions. Here we skip the representation step and directly model the distribution over extrema. To this end, we devise a non-parametric conjugate prior based on a kernel regressor. The resulting posterior distribution directly captures the uncertainty over the maximum of the unknown function. We illustrate the effectiveness of our model by optimizing a noisy, high-dimensional, non-convex objective function.Comment: 9 pages, 5 figure

Language Modeling Is Compression

It has long been established that predictive models can be transformed into lossless compressors and vice versa. Incidentally, in recent years, the machine learning community has focused on training increasingly large and powerful self-supervised (language) models. Since these large language models exhibit impressive predictive capabilities, they are well-positioned to be strong compressors. In this work, we advocate for viewing the prediction problem through the lens of compression and evaluate the compression capabilities of large (foundation) models. We show that large language models are powerful general-purpose predictors and that the compression viewpoint provides novel insights into scaling laws, tokenization, and in-context learning. For example, Chinchilla 70B, while trained primarily on text, compresses ImageNet patches to 43.4% and LibriSpeech samples to 16.4% of their raw size, beating domain-specific compressors like PNG (58.5%) or FLAC (30.3%), respectively. Finally, we show that the prediction-compression equivalence allows us to use any compressor (like gzip) to build a conditional generative model

Meta-learning of Sequential Strategies

In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundations of this tool for building new, scalable agents that operate on broad domains. To do so, we present basic algorithmic templates for building near-optimal predictors and reinforcement learners which behave as if they had a probabilistic model that allowed them to efficiently exploit task structure. Furthermore, we recast memory-based meta-learning within a Bayesian framework, showing that the meta-learned strategies are near-optimal because they amortize Bayes-filtered data, where the adaptation is implemented in the memory dynamics as a state-machine of sufficient statistics. Essentially, memory-based meta-learning translates the hard problem of probabilistic sequential inference into a regression problem.Comment: DeepMind Technical Report (15 pages, 6 figures

Bounded rationality, abstraction and hierarchical decision-making: an information-theoretic optimality principle: supplementary code (v1.0.0)

<p>Release of supplementary code and notebooks for<br> Genewein T., Leibfried F., Grau-Moya J., Braun D.A. (2015): <em>Bounded rationality, abstraction and hierarchical decision-making: an information-theoretic optimality principle</em>, Frontiers in Robotics and AI.</p> <p>Up-to-date versions of the code are found on the corresponding github repository:</p> <p>https://github.com/tgenewein/BoundedRationalityAbstractionAndHierarchicalDecisionMaking</p> <p>The code was tested with Julia 0.3.11 and Jupyter 0.4.1. The notebooks work fine on Chrome, but there are some issues with interactive widgets on Firefox and Internet explorer. See Readme.md for more details on specific Julia package versions and how to get the notebooks to run.</p> <p>The code provided is a proof-of-concept implementation: it is not intended for production nor is it meant to provide a basis library.</p

Data from: Occam's Razor in sensorimotor learning

A large number of recent studies suggest that the sensorimotor system employs probabilistic models to predict its environment and makes inferences about unobserved variables in line with Bayesian statistics. One of the important features of Bayesian statistics is Occam's Razor - an inbuilt preference for simpler models when comparing competing models that explain some observed data equally well. Here we test directly for Occam's Razor in sensorimotor control. We designed a sensorimotor task, where participants had to draw lines through clouds of noisy samples of an unobserved curve generated by one of two possible probabilistic models - a simple model with a large length scale, leading to smooth curves and a complex model with a short length scale, leading to more wiggly curves. In training trials participants were informed about the model that generated the stimulus so that they could learn the statistics of each model. In probe trials participants were then exposed to ambiguous stimuli. In probe trials where the ambiguous stimulus could be fit equally well by both models, we found that participants showed a clear preference for the simpler model. Moreover, we found that participants' choice behavior was quantitatively consistent with Bayesian Occam's Razor. We could also show that participants' drawn trajectories were similar to samples from the Bayesian predictive distribution over trajectories and significantly different from two non-probabilistic heuristics. In two control experiments, we show that the preference of the simpler model cannot be simply explained by a difference in physical effort or by a preference for curve smoothness. Our results suggest that Occam's Razor is a general behavioral principle already present during sensorimotor processing